—. 


: e a Fol ¥ 
“s4 xs. 


rachna program 


y} 2001 1-2006 


women and child heatth 2 at tseale a oe 


working paper series 


paper / 


a ae Pt: Ek es as ci | Se 9 ne ae A eee.” 7 ag ia ae chine a> 
' * SE, la Leal | aida coins, > Sak 4 ll MRE ae m Rite 9 | | 
pg TaN y  ¥ es ; . ape J 
, < : | ) 


————————————  — 1103§°> 


Widening Coverage of Primary Immunization 
De Oe 


Abstract 


Background and Interventions 


Strengthening primary immunization coverage was one of the key components of INHP-II, as part of a package 
of interventions aimed at reducing malnutrition and mortality in children at scale. The unique strength of the 
INHP approach was the strengthening of operational linkages between the national immunization program 
implemented by the Health Department and the ICDS program, which is also mandated to support universal 
immunization. One specific approach was the institution of Nutrition and Health Days (NHD), where a monthly 
fixed-day, fixed-site immunization session is held at the Anganwadi Center (AWC) of the ICDS program, and 
take home rations of supplemental food are distributed on the same day. Other specific strategies included the 
strengthening of delivery systems (such as mechanisms for tracking individual children, the use of “due” lists, 
provision of timely information about immunization sessions, etc), strengthening of program management 
(such as the strengthening of multi-stakeholder review forums at sector, block and district levels, methodical 
supervision, redrawing of boundaries to ensure that supervisory areas of health and ICDS programs coincide, 
etc), capacity building at various levels, complementing efforts of partner agencies in improving quality of 
services, use of data to drive change, and the mobilization of community groups and volunteers to support the 
program. All interventions were implemented at the full project scale of 78 districts across nine states. The 
paper aims to assess the progress in primary child immunization coverage over the life of INHP-II (from 2001 to 
2006) and examines the probable determinants influencing coverage. 


Methods 


Evidence is drawn mainly from state-level estimates of indicators from program baseline and endline surveys, 
district-level estimates from three rounds of periodic assessments from one district in each of the eight 
project states Information from these large-sample surveys was supplemented with process information from 
the program management information system. All estimates presented in the results pertain to INHP-assisted 
regions within the respective states and districts. 


Results and Discussion 


At the aggregate program level, complete immunization coverage among 12-23 month olds increased by 15.9 

percentage points over the project life to reach 58 in almost 
all antigen coverage rates across all states. The la 
immunization coverage increased to 64 percent fr 
measles vaccine showed the largest increase, at 2 
a single vaccine decreased from 19 percent to 10 
Uttar Pradesh and Chhattisgarh continue to have 
immunization services. The proportion of children — 
course reduced by a quarter between the baseline _ 
different states. Compared to state-wide estimat 
assisted areas in seven of eight states had achiev 
from periodic assessments in selected districts sug 
happened mostly after 2003, when INHP int 


Background 


The RACHNA Program The RACHNA program of CARE India included two USAID-supported projects: 
the second phase of Integrated Nutrition and Health Project (INHP-II), which 
focused on child health and nutrition, and the Chayan project, which supported 
interventions for promoting birth spacing and the prevention of transmission 
of HIV/AIDS among groups at high risk. INHP-II, built upon the lessons and 
experiences of the first phase, was implemented in 747 Integrated Child 
Development Services (ICDS) blocks* in 78 districts across nine states* from October 
2001 to December 2006 to complement the maternal and child health and nutrition 
(MCHN) efforts of the ICDS and the Ministry of Health and Family Welfare (MoHFW) 
programs. To achieve its goal of “sustainable improvements in the nutrition and 
health status of seven million women and children”, INHP-II adopted a two-track 
approach - supporting service providers to improve the quality and coverage of 
MCHN services and systems and engaging communities to support better infant 
feeding and caring practices and sustain activities for improved maternal and child 
health and survival. The implementation was facilitated by small program teams of 
CARE, located at the district, state and national levels, and working closely with 
the functionaries of the ICDS program and the programs of the MoHFW, and with a 
range of partners, including local NGOs and Community-Based Organizations (CBOs). 
The main strategies were strengthening of existing systems, behavior change 
communication and capacity building. A detailed description of the program can be 
found in the paper, Program Descnption, in this series. 


Role of RACHNA In providing immunization services, the RACHNA program focused primarily on 
in Primary improving the operational effectiveness of the ICDS program and the national 
Immunization immunization program of the MoHFW by promoting functional convergence 


between the two. Thus, the results and lessons described in this paper are directly 
relevant to both these national programs and related stakeholders. 


The district and state teams of RACHNA provided techno-managerial guidance 
and support to ICDS and Health department functionaries primarily at block and 
lower levels to maximize immunization coverage. Immunization interventions 
were integrated with other elements of child health. The broad components of 
support were: 


e Strengthening delivery systems to maximize coverage 
e Strengthening program management 

e Training and capacity building 

© Cold chain, logistics and injection safety. 

e Monitoring 

© Social Mobilization/Behavior Change Communication 


Box 7.1 elaborates the various activities under each of these broad components. 


1 A block is an administrative sub-unit of a district, having a population of around 100,000, and often larger. An ICDS block is 
usually identical to the administrative block, and comprises about 100 Anganwadi Centers (AWC). Not all blocks in a district and 
not all villages and hamlets within a block may be served by the ICDS program. INHP-II was designed to be implemented only 
in ICDS-served blocks, but did not necessarily cover all ICDS blocks in the districts where it was implemented. 


2 These include Andhra Pradesh (AP), Bihar (BI), Chhattisgarh (CG), Jharkhand (JH), Madhya Pradesh (MP), Orissa (OR), 
Rajasthan (RA), Uttar Pradesh (UP) and West Bengal (WB). Since the program was implemented in Bihar only from late 
2004 onwards, results presented are only from the other eight states. 
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Box 7.1: Key Interventions and Approaches for Improving Immunization 


Strengthening delivery systems to maximize coverage 
® Encourage use of social maps and routine surveys of Anganwadi Workers (AWWs) and Auxiliary Nurse Midwives 
(ANMs) to ensure that every community and family is brought into the net of services 


e —Link-up pregnancy and child service registers to ensure inclusion of every birth in service registers 


© Draw up and use lists of children due for immunization 


® Use home visits to inform families in advance about an impending immunization session. 


Strengthening program management 

© Coordinate with Health department to plan and execute the fixed-day, fixed-site outreach immunization 
sessions 

® Help the ICDS and Health departments to coordinate the immunization sessions in conjunction with the 
distribution of food rations from the AWC (these joint sessions were termed Nutrition and Health Days or NHD) 


® Help the Health department to optimize the utilization of available vaccinators (ANMs, Lady Health Visitors 
and at times Multi-Purpose Health Workers (male workers) to cover all areas) 


® Support vigilant review and monitoring of these efforts at two crucial levels - ICDS sector meetings with ANMs 
in attendance, and at the block level forums with the joint participation of the ICDS, Health and Panchayat 
representatives 


® facilitate the use of tracking mechanisms based on best available denominators at each level 


* Support the ICDS and Health departments in coordinating managerial efforts to redraw boundanes of 
field workers and their supervisors to ensure the convergence of administrative units of the two programs 
converged, thus minimizing discrepancies in reported figures. 


Training and capacity building 
* Conduct one-time brief joint training of AWW/ANM, with a focus on management and quality areas 


* Utilize available opportunities in the routine implementation schedules of ICDS and health programs to keep 
functionaries updated and focused on priorities. 


Cold chain, logistics and injection safety 


* Complement efforts of agencies such as UNICEF and WHO in minimizing breakdowns of supply and distribution 
lines 

* Encourage use of standard protocols for matters such as cold chain maintenance, the practice of policies of 
opening vaccine vials even when only a few children are present, and basic injection safety protocols. 

Monitoring 

> 


Facilitate periodic coverage assessments (rapid surveys) at district level to help program management 
* Support on-going ass ent of quality of immunization sessions through session observations 
* Facilitate review of immunization (sessions, quality, coverage) data in block/district forums. 


Social mobilization/BCC 


* — Encourage Change Agents (CAs) community health volunteers, community based organizations (CBOs) and 
members of the Panchayat to attend, support and monitor the NHD 

* Mobilize CBOs (women’s groups) to disseminate information 

> 


Support communication campaigns involving multiple channels and media including folk media. 
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While the program complemented efforts such as the promotion of rigorous 
cold chain maintenance and promotion of injection safety procedures, the 
primary focus of RACHNA interventions was on maximizing coverage of primary 
immunization through some or all of the components listed above. 


In practice, there was considerable variation in the intensity of implementation 
of these components across states, districts and blocks, and time. By design, 
the interventions reached all program areas only in the last year of the program. 


This paper uses available evidence to assess the progress in immunization coverage 
over the life of the program, examine the probable determinants influencing 
immunization coverage and draw lessons for making the national immunization 
program more effective. It focuses on primary child immunization alone. Since 
Hepatitis B immunization was introduced into the national immunization program 
later, it was not a focus of RACHNA, or of this paper. Maternal Tetanus Toxoid coverage 
and related issues are dealt with in the paper, Enhancing Newborn Care, in this senes. 


Methods of Assessment 


The RACHNA program had three kinds of quantitative assessments featuring 
household surveys: (i) pairs of baseline and endline assessments; (ii) periodic 
rapid assessments in a panel of one district from each of the eight program states; 
and (iii) two evaluation research studies in two districts. In all surveys, the AWC 
was the Primary Sampling Unit (PSU), and respondents for the primary household 
level interviews were mothers of children 0-23 months of age. 


Baseline and Endline Surveys 


all 


The endline survey of INHP-I (early 2001) served as the baseline survey of 
INHP-II. The INHP-I endline for Bihar served as the baseline for Jharkhand and 
the Madhya Pradesh endline served as the baseline for both Madhya Pradesh and 
Chhattisgarh. The INHP-I endline for Bihar served as the baseline for Jharkhand 
and the Madhya Pradesh endline served as the baseline for both Madhya Pradesh 
and Chhattisgarh, since the new states of Chhattisgarh and Jharkhand were 
created from a division of the erstwhile Madhya Pradesh and Bihar respectively, 
just prior to the INHP-endline survey.’ 


INHP-I consisted of three kinds of program areas based on intensity of interventions 
and effort: the “High impact” blocks, “Capacity building blocks” and “Other 
blocks”. The 2001 survey was designed to generate separate estimates of these 
three areas through a multi-stage sampling design: a fixed number of blocks and 
PSUs was randomly picked from each of the three areas. A total of 540, 540 and 
— respondents (mothers of children 0-23 months old) were selected respectively 
using a predetermined random selection process. The interview tool, common for 
all children between 0-23 months, covered all interventions - antenatal, natal and 
newborn care, infant feeding and immunization. State-level estimates, derived by 
applying population weights to the three areas, are used for all comparisons with 
the endline, without reference to the three kinds of program areas. 
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The endline survey of INHP-II (early 2006) was used as a multistage sampling 


design, but this differed in some respects from the baseline. The respondents 


(mothers of children between 0-23 months old), were drawn from two groups. The 


mothers of children 0-5 months of age were asked questions related mainly to 
antenatal, natal and newborn care and breastfeeding, while mothers of ides 
6-23 months of age were interviewed with questions mainly related to 
complementary feeding and immunization. This helped minimize recall bias 

and capture more recent events, likely to have been influenced by program 
interventions. The sample size was sufficient to detect a ten percentage point 
difference in an estimate with 95 percent confidence levels and 80 percent power 
and an assumed maximum design effect of 1.8. The numbers of PSUs and blocks | 
selected in each state varied according to the birth rates, being higher in states 
with lower birth rates. Blocks were selected in a manner that ensured proportionate 
representation of urban, rural and tribal blocks, and PSU selection within a block 
ensured proportionate representation of demonstration sites,’ replication sites and 
other sites. Sampling frames were generated for children in the age group of 

0-5 months and 6-23 months by prior house-listing and the target sample picked 
by circular systematic sampling, making allowance for a non-response rate of 

15 percent. For each group, the target number to be completely interviewed was 
733. Effectively, this resulted in a virtually self-weighted sample for each state. 


Periodic Rapid Assessments (RAPs) in the Panel Districts 


In order to monitor progress in outcomes to inform program strategies, a panel 
of one district from each of the eight states was established in 2003, where 
three rounds of periodic assessments were conducted between 2003 and 2005 at 
approximately annual intervals. The universe for these assessments was the first 
phase replication sites (the first batch of 25 percent AWCs in the district where 
at-scale implementation began). 


Mothers of children 0-5 months of age were interviewed on antenatal, natal and 
newborn care and breastfeeding, while mothers of children 6-23 months of age 
were interviewed on complementary feeding and immunization. Round 1 had a 
two-stage design, first randomly selecting five blocks from each district, and then 
five PSUs from each block, followed by selecting a fixed number of 0-23 months 
old from each PSU, whose mothers were respondents. The target sample size was 
150 for children 0-5 months old and 450 for children 6-23 months old. Rounds 2 
and 3 used one-stage design, directly picking 90 PSUs from the universe, spread 
across all blocks in the district, and then randomly selecting the target sample 
(460 each for the two age groups of 0-5 and 6-23 months) from a sampling 
frame generated by house-listing after allowing for a 15 percent non-response. 
The latter samples were sufficient to detect a difference of 10 percentage points 
in estimates of two surveys with 95 percent confidence and 80 percent power, 


ation Site (DS) was an AWC that held an NHD every month, had ; 
ties, had at least three active Change Agents and had other need- 
rved to demonstrate how these “best practices were 
district. The AWCs beyond DS that were reached by 

de a sense of the duration of RACHNA 


3 As described in the paper, Program Description, a Demonstr 
community group that managed nutrition and health activi 
based innovations. The DS were largely supported by local NGOs, and se 
to be implemented, thus facilitating their replication to the rest of the 
end-2004 were termed Replication Sites (RS) and the rest as “Others”, in order to prov! 


interventions in different AWC. 
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assuming a small design effect. The estimates from the first round were expected 
to be less precise, particularly for the smaller sample of the 0-5 month group. The 
tools used in Round 1 were modified to add more questions and refine existing 


ones, while ensuring maximum comparability. 


Comparisons of Coverage Rates with Other Surveys 


National Family Health 
Survey-2 (NFHS-2) and 
National Family Health 
Survey-3 (NFHS-3) 


Coverage Evaluation 
Surveys (CES) 


Comparisons between NFHS-3 and INHP surveys are based on the premise 

that, for indicators that have comparable definitions, NFHS estimates effectively 
represent the weighted average of INHP-served and non-INHP-served areas of a 
given state, and that differences between these areas are primarily on account of 
INHP interventions. In reality, other factors could play a role in causing these areas 
to differ; such potential factors are described in the paper where relevant. 


The NFHS and the INHP surveys do differ slightly in the way immunization data 
is collected and analyzed. In the NFHS, after having recorded the information 
from the immunization card, the respondent is asked, referring to each antigen 
and dose, whether it was administered. If a vaccine has not been recorded in the 
card but is reported by the respondent, the information collected from the card 
is accordingly modified, using separately identifiable codes. For respondents who 
do not produce an immunization card, they are asked to recall information about 
the coverage of antigens which is recorded separately. In the INHP surveys, card 
data is retained as recorded in the card, while recall information is also recorded 
separately, without reference to the card.-However, when estimating “either 
source” coverage figures (which are used in this paper), recall information is 
given preference over card data in both surveys, and it is unlikely that the INHP 
and the NFHS methods would produce significantly different estimates. 


In the Coverage Evaluation Surveys (CES) conducted by UNICEF, card data is 

not separately recorded, but cards are used to supplement recall information 
obtained from respondents, approximating the “either source” estimates of the 
NFHS. A large number of process questions related to immunization are asked, but 
available reports do not carry significant determinants analysis. As in the case of 
NFHS, CES is also designed to cover all areas, not just the ICDS coverage areas. 
There is limited use of CES data in this paper. 


In tabulating and presenting results, estimates of indicators are presented 
separately for each district or state as the case may be, and in most cases, the 
average program-wide estimates are not emphasized. This pattern has been 


followed to retain the focus on individual states and districts, among which there 
is considerable variability. 


Also, statistical significance testes are not presented for most primary results, 
such as when comparing estimates for indicators across baseline and endline 
Surveys, or across rounds of RAPs. Most of the surveys were large sample surveys, 
designed to detect differences of 10 percentage points or more between two 
comparable rounds. While confidence intervals or p-values could have been 


6 _—RACHNA Program 2001-2006 


presented, this would have made the already large tables, each bearing results 


from eight states or districts, even less user-friendly. Instead, the auth " 
taken the view that it is safe to assume that a difference of 10 uae ave 
or more between rounds is likely to be statistically sliantlicant tn minat TS 
that showing statistical significance for differences of less than 10 borin: 4 
points may not be convincing from a program perspective. Thus, deaeieaiae of 
results also generally distinguish between differences of 10 percentage points 

or more (as being statistically significant and programmatically relevant in most 
cases), and lesser differences (as being not convincing in most cases). While this 
approach oversimplifies the presentation of results, it should help the general 


reader interpret results more easily. More experienced and interested readers will 
look deeper, in any case. 


Results and Discussion 


In practice, there was considerable variation in the intensity of implementation 
of these components across states, districts and blocks, and over time. By design, 
the interventions reached all program areas only in the last year of the program. 


Changes in Immunization Coverage between Baseline and Endline Surveys 


Table 7.1 presents changes in individual vaccine coverage rates from the baseline 
to the endline, including the state-specific and weighted* program-wide coverage 
estimates, by card or recall (“either source”) as well as left-out and drop-out rates." 


At the aggregate program level, complete immunization coverage went up 

15.9 percentage points to reach 58.2 percent by the endline. Among individual 
antigens, measles vaccination showed the largest increment (20.5 percentage 
points) to reach about 68 percent by endline, followed by DPT3 (about 14 
percentage points increment to reach 72 percent), OPV (about 13 percentage 
points increment to reach 70 percent), and BCG (9 percentage points increment to 
reach 88 percent). All of these are statistically significant differences.” 


While each state showed increments from the baseline, the variation across states 
was wide. For instance, increases in complete immunization coverage ranged from 
a low 4.2 percentage points in Chhattisgarh to a remarkable 36.3 percentage 
points in Jharkhand. Uttar Pradesh was the only other state that showed less than 
10 percentage point increment. Except for Andhra Pradesh, however, complete 
immunization rates remain well below the 80 percent mark. 


Similar increments were seen in the case of individual antigens. The ranges of 
increments in the coverage of different antigens at the aggregate program level 
were: BCG: 0.5 to 35.8 percentage points; DPT3: 1.7 percent to 36.4 percent; 
OPV3: 4.1 to 34.9 percentage points; and Measles: 8.9 to 54 percentage points 


+ Weighted by the relative population size representing each state. ; Ae 
| 3 seats ‘: noted that only about a third of all respondents were able to show interviewers an delineate 
concerned child, and thus these rates are heavily based on the recall of the respondents. West inary Ag bo i the 
with more than 80 percent producing an immunization card; Orissa (57.4 percent) and Andhra Pradesh (47.9 P 
only other states where a significant contribution could have come from card data. 
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(Table 7.1). The largest increment in the case of each antigen was seen in 
Jharkhand. Although Jharkhand contributed about 15 percent weight to the 


program average across states, the program-level increments cannot be attributed 
primarily to changes in this state alone. 


In the case of Chhattisgarh/Madhya Pradesh and Jharkhand/Bihar, there have 
been shifts in the program universe since the baseline, and esis for these stat 
should be interpreted with caution. Please see endnotes’ and the discussion in % 
the Methods Used for Assessments paper for details of the issues involved. 


Among achieved coverage rates at the endline, DPT3 coverage is higher than 
measles coverage in all states other than Jharkhand. In most states. OP\/3 
coverage tends to trail DPT3 coverage by small margins. 


Program-wide, the proportion of children not receiving a single vaccine 
(completely left out of immunization services) reduced by about a third from 
about 19 percent at the baseline to about 10 percent at the endline. Rajasthan 
(25.4 percent), Uttar Pradesh (16.9 percent) and Chhattisgarh (15.2 percent) 
continue to have substantial left-outs even at the endline, while the rest of the 
states have less than 10 percent left-outs - Andhra Pradesh (2.2 percent) and 
Jharkhand (2.7 percent) have the least left-outs (Table 7.1). Jharkhand had the 
most impressive reduction in left-outs, beginning as the state with the highest 
proportion of children completely left out at the baseline (39 percent), to one 
with the lowest left-out proportions at the endline. Other states showed only 
small change in the proportion of children left out of immunization coverage. 


The vaccine drop-out rates among children represent the proportion of children 
who start receiving vaccines, but do not get one or more of the remaining 
vaccines in the schedule offered by the National Immunization Program. Table 7.1 
presents drop-outs estimated in two ways: 


1. The conventional indicator, BCG to measles vaccine drop-out, is expressed as 
the proportion of children receiving BCG vaccines who had yet not received 
measles vaccine by the time they were interviewed in their second year. 
Program-wide, at the baseline, about 39 percent of those who received BCG 
had failed to receive measles vaccine. At the endline, this proportion was 
almost halved to about 22 percent. Every state contributed to this reduction 
in drop-outs, ranging from about three percent in Uttar Pradesh to 36 percent 
in Jharkhand (Table 7.1). 


2. Amore comprehensive estimate of drop-outs is the children who failed to 
complete all vaccines expressed as a proportion of children who received 
any vaccine. This is presented in Table 7.1 as “true” drop-outs. Again, there 
are clear reductions in drop-outs across all states, with Jharkhand and West 
Bengal showing the largest reductions of about 20 percent points from 
baseline to endline. Program-wide, drop-outs have been reduced by about a 
quarter (48 percent to 35 percent). 


However, drop-out rates remain substantial across most states even at the 
endline, including in those states that apparently performed well, such as 
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Immunization coverage 
in INHP-II areas at the 
endline, compared to 
NFHS-3 and CES 


Figure 7.1: DPT3 coverage 
comparisons for RACHNA 
program states - CES (rural, 
2005), NFHS-3 (rural, 2006) 
and RACHNA Endline (2006). 


Jharkhand, and “true” drop-out rates range from 16 percent in Andhra Pradesh to 


49 percent in Chhattisgarh at the endline. 


In sum, there have been significant and programmatically relevant increases 
in antigen coverage rates across all states over the life of the program. At the 
same time, there are substantial variations across states, in terms of both the 
magnitude of increments and achieved coverage rates. Among states that had 
relatively lower coverage at the baseline, rates still remain below national 
program objectives with the exception of Jharkhand. 


The latest round of CES survey data was collected in late 2005, the NFHS-3 in 
2005 to 2006, and the INHP-II endline survey was conducted between February 
and March 2006, making them all roughly contemporaneous. Comparisons of 
coverage rates across these three surveys are presented here. Changes between 
the NFHS-2 and NFHS-3 (provisional estimates) and the changes between 
baseline and endline surveys of INHP-II are also examined to determine possible 
differences in the rate of change in the areas represented by these pairs of 
surveys. The time-span between the National Family Health Survey (NFHS-2) 
and 3 is approximately eight years and that between the baseline and endline 
surveys of INHP-II approximately five years and this is factored into estimating 
the rate of change. Also, it is worth noting that since the INHP-II efforts 
reached their full extent of implementation only in the last year of the program, 
it is likely that the pace of change accelerated during the latter years of the 
program, rather than being uniform over the life of the program. 


Figures 7.1, 7.2 and 7.3 compare coverage estimates of CES (rural) 2005, NHFS-3 
(rural) and the RACHNA endline for each state for DPT3, measles and complete 
vaccination, respectively among children 12-23 months of age. 


e With the exception of measles vaccine coverage in Chhattisgarh, the RACHNA 
endline estimates for coverage are consistently higher than NFHS-3 (Rural). It 
appears that higher DPT3 coverage rates are largely responsible for the large 
differences in complete immunization rates between NFHS-3 and INHP-II. The 
differences in measles coverage rates are comparatively smaller. 


% children 12-23 months 
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Figure 7.2: Measles coverage 
comparisons for RACHNA 
program states - 

CES (2005), NFHS-3 (2006) 
and RACHNA Endline (2006). 


Figure 7.3: Complete 
immunization coverage 
comparisons for RACHNA 
program states - 

CES 2005, NFHS-3 (2006) 
and RACHNA Endline (2006). 
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e The comparison with CES 2005 figures for DPT3 and measles vaccines is not as 
consistent, being lower in INHP areas in some states and comparable or higher 
in others. In the case of complete immunization, however, the INHP-II endline 
estimates are at least 10 percentage points higher than CES 2005 figures in four 
of the eight states (Andhra Pradesh, Jharkhand, Orissa and West Bengal), while 
the differences in the rest of the states are less than 10 percentage points. 


Figures 7.4 and 7.5 compare state-wide increments, as estimated by the 
difference between NFHS-2 and NFHS-3 and the differences between baseline 
and endline.estimates in INHP-II areas of the same states. The increments 
across the pairs of surveys are expressed as a percentage of the respective 
baseline, to partly compensate for differences in baselines. In the case of 
three of the eight states (Andhra Pradesh, Orissa, Rajasthan), the total 
(Figure 7.4) and average annual increments (Figure 7.5) in inte ae areas 
appear to exceed the respective state-wide increments as indicated by NFHS. 
In the case of Jharkhand, Madhya Pradesh, Uttar Pradesh and West Bengal, the 
pro-rated annual increments state-wide and in INHP-supported areas appea! 
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Figure 7.4: Full 
immunization coverage: 
Percent increment 
between surveys - 
NFHS-2-3 vs CARE 
Baseline-Endline. 


Figure 7.5: Full 
immunization coverage: 
Annual percent 
increment between 
surveys - NFHS-2-3 vs 
CARE Baseline-Endline. 


Percent increment, children 12-23 months 


MINFHS percent increment MM CARE percent increment 


Annual percent increment, children 12-23 months 


JH MP OR RA 


MNFHS percent increment M# CARE percent increment 


about equal (the differences being less than five percentage points annually), 
while in the case of Chhattisgarh, state-wide increments exceed increments in 
INHP-II areas. 


Vaccination drop-out rates, exemplified by the BCG and measles drop-outs 
presented in Figure 7.6, also follow similar patterns. As can be seen, the 
reductions in drop-outs in INHP-supported areas between baseline and endline 
surveys have exceeded reductions in drop-outs state-wide (as represented by 
NFHS) in all states other than Chhattisgarh and Madhya Pradesh. As indicated 
earlier, some of these results need careful interpretation in the light of the known 


shifts in program universe since the baseline, for both, INHP and NFHS (Please 
see endnote’). 


Since estimates for children not receiving a single vaccine are not yet available 
for the NFHS-3, it is not possible to present comparisons on either complete left- 
outs or for “true” drop-outs. Similarly, since estimates of timely immunization 
have not been released for the NFHS-3, it is not possible to present any 
comparable estimates for this indicator. 
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Figure 7.6: Reduction in 
BCG-Measles dropout 
percentage, NFHS-2-3 vs 
CARE Endline-Baseline. 
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In sum, immunization coverage appears to have increased faster and reached 
higher levels across INHP-II areas, as compared to other areas in the respective 
states in most instances. 


One clear and well-recognized pattern that emerges from the results is that 
program areas vary widely, both in terms of the coverage and change over time. 


The state-level variations were not always predictable, even by field teams. The 
large increases in Jharkhand, for instance, appear to have been achieved by 

very recent intense catch-up rounds that happened in the state (starting 2005). 
These campaigns, owned by all major stakeholders and development partners, 
were focused on micro-planning and execution. This alone was not expected to 
produce very significant results, given the rather weak health infrastructure in the 
state. The uncertain baselines for this state, for both the INHP-II program and 
for NFHS, make the magnitude of change a little uncertain, but coverage by the 
endline was certainly higher than expected, by any measure. Even as recently as 
2004, the projected coverage rates were in the order of 20-40 percent, which was 
borne out by the RAPs results from Lohardaga, the panel district in Jharkhand. 
One explanation for the rather unusual differences seen between NFHS-3 and 
INHP endline levels of achieved coverage could be that, despite covering most 
districts in the states, the RACHNA program covers only about half the ICDS 
blocks in the state and probably does not cover some of the most difficult blocks 
in many districts. The larger improvements in INHP-II areas as seen in the endline 
appear to have been achieved because the field presence of RACHNA teams in 
“better” districts further boosted the capacities of local teams to achieve results 
through catch-up rounds. RACHNA’s focus on improving nearly-absent tracking 
systems, including simple service registers, could have contributed significantly. 
Again, this clearly establishes how even a marginally stronger system can achieve 
more if given critical support. Whether Jharkhand can sustain these remarkable 
achievements will certainly depend on whether these changes have been achieved 
on the strength of improved delivery systems. 


The high level of achieved coverage rates in Andhra Pradesh and West pg i 
are undoubtedly a function of strong systems and many years of focused efforts, 
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as well as of communities which have learnt to accept and expect immunization 
as a part of the normal growth of children. Field observations attest to the 
strong recording and tracking systems in place, as well as responsible, focused 
leadership. In West Bengal, in particular, the strong role of the Panchayat in 
monitoring both health and ICDS programs is significant. In both states, INHP 
helped strengthen existing local approaches, encouraged data use and review and 
promoted convergence between ICDS and Health. 


In Orissa, more than in any other state, the increases in complete immunization 
rates can be largely attributed to increases in measles immunization. While 
individual antigen coverage rates are around 75 percent or higher, complete 
;mmunization coverage is significantly lower, indicating that there are gaps in 
tracking systems. On the ground, since Orissa ICDS and Health field teams appear 
to have relatively strong recording and reporting systems, it seems likely that 
actual follow up of individual children is not adequately supervised. The story of 
Madhya Pradesh is similar, but it lags behind on measles coverage. 


In Chhattisgarh, the relatively high left-out proportion, the unusually large gap 
between DPT3 and OPV3 (possibly a measurement error) as well as the poor 
measles coverage have probably contributed to the low complete immunization 
coverage in spite of the apparently rapidly falling mortality rates (SRS, 2003 to 
2005), and of the state having enthusiastic leadership that is supporting several 
innovative approaches and community level interventions. 


In Rajasthan, the relatively fast increase in coverage rates holds promise, 
although absolute coverage rates are still very low. Of late, the state government 
has been very strongly pushing for improved routine immunization. INHP efforts 
have evidently not led to significant change due to fundamental weaknesses in 
the system and poor accountability structures in both Chhattisgarh and Rajasthan 
that set them apart from states such as Andhra Pradesh, West Bengal and 

Orissa. Additionally, in Uttar Pradesh, the influence of the pulse polio program 

on routine services is felt maximally and indeed leads to a vicious cycle, since 
one of the reasons for its failure to eliminate polio is the poor penetration of 
routine immunization services. RACHNA efforts have evidently been inadequate to 
substantially overcome fundamental system and design defects here. 


There is limited data available from independent surveys to document variation 
within a state. The best illustration comes from the comparison of the 
performance of the “panel” district and that of the corresponding state. The 
three rounds of periodic rapid assessments (RAPs) conducted in one district in 
each of the states in 2003, 2004 and 2005 provide the opportunity to make such 
comparisons with the state-wide baseline/endline surveys. 


Figure 7.7 compares complete immunization coverage over three rounds in 

the panel districts and the coverage in the corresponding state as seen in the 
endline survey in early 2006. While in third round of RAPs, district figures are 
close to state figures in Andhra Pradesh, Orissa, Uttar Pradesh and West Bengal, 
the differences in other states exceed 10 percentage points, with the state level 
coverage rates being higher except in the case of Chhattisgarh. Also of interest 


RACHNA Program 2001-2006 


Figure 7.7: Trends in 
complete immunization 90 
coverage in the panel 80 
district assessments 
Rounds 1-3 (2003, 2004, 
2005), and state-level 
Endline (2006). 
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are the annual fluctuations at the district level, such as in Andhra Pradesh, 
Madhya Pradesh and Rajasthan. 


This variation across districts (and possibly across sub-district regions, although 
not measured in the program) over time probably represents the complex 
influence of variation in factors such as socio-demographics, geographical 
access, and intensity of INHP efforts - the last being particularly prone to 
changes over time. Some of these have been referred to above, and some, 

for which survey evidence is available from the INHP, are dealt with in the 
subsequent sections. 


Factors Influencing Change in Immunization Coverage 


A number of program and non-program factors are known to affect immunization 
coverage rates, from program experience nationally and globally. The Universal 
Immunization Program (UIP) review of 2004 identified and highlighted a large 
number of deficiencies in the national immunization program and made concrete 
recommendations for structural and operational reform. Several of the measures 
recommended were already in use in the INHP program areas. While no specific 
studies were conducted to directly measure the effect of individual processes 

and factors on immunization coverage, available information from the INHP-II 
baseline and endline surveys, the RAPs and the Health Management Information 
Systems (HMIS) have been analyzed to understand the influence of several 
variables likely to influence immunization coverage. This analysis is done more 
from a programmatic perspective, rather than an academic one. It should be noted 
that due to lack of baseline information on some of the variables discussed below, 
a comparative analysis of change over time has not been possible. 


The influence of NHD INHP promoted the systematic conduct of monthly Nutrition and Health es 7 
and food rations (NHDs) to increase health service coverage. These served as : — to prac a 
converge ICDS and Health services and combined the immunization and antena 


, ° 
care services provided by the ANM and the take-home rations given out on the sam 


Widening Coverage of Primary Immunization 15 


fixed-day once a month at the AWC. NHDs provided the attraction of supplementary 
food rations in addition to the known benefits of a fixed-day, fixed site approach to 
outreach services for child immunization as well as antenatal care. 


Considerable effort was involved in coordinating the ICDS and Health Department 
program field schedules, such that the distribution of take-home rations (THR) at 
the AWC for pregnant and lactating women and children under three years of age 
took place on the same day as the visit of the ANM to the village for the provision 
of immunization and ANC services. In states where outreach services of ANMs 
were well-established, it involved re-doing the field visit rosters of ANMs and 
rescheduling THR days by ICDS. In some states, where the concept of outreach 
services by ANMs was itself new, it involved greater effort. Not all states at all 
times were able to fully institutionalize NHDs. 


The monthly NHDs were monitored and by the endline, about two-thirds of all 
AWWs were reporting that NHDs were happening every month (defined as THR and 
either immunization or antenatal care by the ANM happening on the same day in 
the month at the AWC). Antenatal check-ups required a minimum degree of privacy, 
which was not available at AWCs, and this matter was never satisfactorily resolved. 
The regularity of food supply heavily influenced the regularity of NHDs. In early 
2003, when the CARE-supplied PL480 Title II grain, CSB (Corn Soya Blend) was 
replaced by state government provided grain in the INHP-II areas, the monthly 
NHDs were affected, at least in the short term, as a result of the transition and 
the associated interruptions in the supply of food supplements. In some places, 
the insistence on hot cooked food for everyone further complicated the picture.° 
The decision to provide food supplements universally to all pregnant and lactating 
women and children six months to six years old, and to extend the ICDS scheme to 
all villages, should effectively make the NHD approach also universal. 


Program monitoring data indicated that by the end of INHP-II, almost 60 percent of 
the AWCs reported conducting regular NHDs. West Bengal remained at about 

50 percent, while Chhattisgarh and Rajasthan reported in excess of 75 percent of 
AWC holding NHD in the last year of the program. The endline interviews of AWW 
also provide a similar picture, presented in Table 7.2. About 90 percent or more 

of AWW in states other than West Bengal reported that fixed-day immunization 

or antenatal care services had been provided at least once in the three months 
preceding the endline interviews. Overall, about two-thirds of them reported that 
such services had been provided in the previous month. In states other than West 
Bengal, about three-fourths or more of AWW also reported that food supplements 
had been distributed on the same day as the fixed-day health services at least once 
in the previous three months. Between 47 percent and 77 percent of AWW reported 
that this had happened in the previous month, indicating that not more than 60 
percent of AWC are able to hold a NHD by definition every month. 


— level of penetration of NHDs could partly explain the increase in 
immunization coverage across most states. Figure 7.8 shows that there is only 


“It should be mentioned here that take-home rations 
these several TH 


for THR as per t 


D eeetions 3 (THR) are given out either weekly, fortnightly or, monthly and only one of 
“al Gaal 's coordinated with fixed-day, fixed-site immunization services. Also, while all families are eligible 
‘stribution schedule, a given infant in a family is due for immunization only on about five days in a year. 


» @ RACHNA Program 2001-2006 


Table 7.2: Regularity of NHD and immunization sessions as reported by AWW, Endline (2006 
; Z ) 


i 


Number of AWWs interviewed at the endline | | 196; 159| 106 re | ur | we | a 
Immunization or antenatal services provided at the AWC: ms " ' |_ 434] 13%] 362) 1229 
at least once in the previous three months + asst ws 92.5| 87.2! 89.2 | 94.0| 86.6 | 
in each of the previous three months 84.2 wat a er | _ oy : _ | 65.2 82.5 
Food supplements distributed on the same day as the services were provided 688 _ =-6 | 73-6 | 604) 62.4] 66.0 
83.1| 84.0| 78.2] 73.9| 88.1] 79.1] 55.5| : 
ro lconr leas asa sa eset pt en 


a rough association between the levels of NHD regularity reported by AWW and 
the complete immunization rates estimated from household surveys, although 

it is not entirely appropriate to draw conclusions from such broad comparisons. 
Field observations reflect variable effectiveness in delivering services on NHD by 
AWWs and ANMs. Several elements such as defining the denominators, systematic 
tracking of individual children by preparing due lists for each session, availability 
of vaccines and opening of vials irrespective of the number of children are also 
critical in influencing immunization coverage rates. 


It is also interesting to note, however, that a large proportion of the AWWs, ranging 
from 53 percent to 81 percent, reported that the ANMs administer vaccines in their 
AWC catchment area on days other than NHDs as well (Table 7.2), making it difficult 
to attribute changes in immunization coverage to NHDs alone. Surprisingly, the 
proportion of villages where ANMs reportedly provided vaccination services on days 
other than NHDs did not differ significantly between villages having or not having a 
health sub-center’ (data not shown in the table), though one might have predicted 
that such extra vaccination day would have been more common in sub-center 
villages. This may indicate that the ANM's roster and coordination with the AWW 
may be significantly less than perfect. 


One expected effect of the NHD approach was that immunization sessions held 
at decentralized locations (the AWC) would increase the proportion of children 


Figure 7.8: Reported 
regularity of NHD compared 
to complete immunization 
coverage, Endline (2006). 
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7 The sub health center is the ANM’s post and official residence. 
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Figure 7.9: Proportion of 
children getting vaccines at 
the AWC, Baseline (2001) 
and Endline (2006). 
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getting immunized at AWCs. Figure 7.9 provides evidence from baseline and 
endline surveys that this has indeed happened over the life of the program. In 
three states where immunization rates were known to be low at the baseline, 
Jharkhand, Rajasthan and Uttar Pradesh, there were large increases in the 
proportion of children being immunized at the AWC, with modest increases in 
most other states. Madhya Pradesh maintained a high of over 75 percent from 
baseline to endline. Only in West Bengal, where immunization happens mostly 
at the sub-health centers and immunization rates are fairly high, has the role of 
the AWC remained marginal (although the AWW may still play a role in mobilizing 
families to seek immunization services). In all other states, the majority of 
children are now getting immunized at the AWC.® 


The relationship between receiving supplementary food from the AWC and vaccine 
coverage rates can be examined in several ways, since questions about the receipt 
of food supplements as well as vaccines were asked to respondents (mothers) in 
all surveys. Figures 7.10a, 7.10b, 7.11a and 7.11b present, for children 12-23 
months old, simple comparisons between the baseline and the endline of reported 
food supplement receipt rates during a particular period of infancy as reported 

by respondents at the endline survey, and of the current coverage of one antigen 
normally given during the same period of recall: 


° Figure 7.10 compares receipt of food at ages less than six months with DPT3 
coverage, at the baseline (7.10a) and the endline (7.10b) across states. At 
the baseline, rates of receipt of food supplements and the rates of DPT3 
coverage diverge widely, and the relationship appears to be inconsistent 
across states. At the endline, rates of food receipt and the rates of DPT3 
coverage are much closer to each other, consistently across states. 


Figures 7.11a and 7.11b compare receipt of food supplements at ages more 
than six months with measles vaccine coverage at baseline and endline across 
states. Patterns are similar to those found in the case of DPT3: food receipt 


Interestingly, this is not true of antenatal check-ups, perhaps because of the lack of privacy referred to earlier. 
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Figure 7.10a: Receipt of 
food supplements at 0-5 
months and DPT3 coverage, 
Baseline (2001). 


Figure 7.10b: Receipt 

of food supplements at 
0-5 months and DPT3 
coverage, Endline (2006). 
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and measles coverage rates appear closer to each other at the endline than at 
the baseline. 


These findings seem to indicate that while food supplementation without any 
explicit convergence with health services was by itself insufficient to drive 
immunization coverage (as seen at the baseline), the NHD approach, which 
emphasized fixed-day, fixed-site outreach services at the AWC, did bring the service 
delivery and communities together and led to an increase in coverage rates. 


A more direct measure of the influence of food supplements on immunization 
coverage rates is presented in Table 7.3. This examines, for the endline, the 
relationship between the receipt or non-receipt of food supplements at different 
periods and complete immunization rates among these two groups for each state. 
As can be seen, except in Andhra Pradesh, the receipt of food supplements during 
at least one of the three periods (pregnancy, child's age less than six months, 
child’s age greater than six months) is significantly associated with the child | 
being fully immunized. The relationship holds strongly across all three perrocs ™ 
the states of Orissa, Rajasthan and Uttar Pradesh. At the same time, it 1s clear 


Widening Coverage of Primary Immunization 19 


Figure 7.11a: Receipt 

of food supplements at 
6+ months and measles 
vaccine coverage, Baseline 


(2001). 


Figure 7.11b: Receipt of 
food supplements at 6+ 
months and measles vaccine 
coverage, Endline (2006). 
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that even food recipients do not have satisfactorily high immunization rates in 
most states,’ and thus other factors must play a significant part. The influence 
on immunization rates of making access to food supplements universal thus 
depends on the current coverage with food supplements, and the differential 
immunization coverage between those currently receiving or not receiving food 
supplements. As shown in the table, in Rajasthan for instance, where coverage 
with food supplements is low, universalization of food supplements, if it can be 
accomplished, is likely to yield an increment of about 11-14 percentage points 
In complete immunization coverage," at the given level of NHD quality. In other 
states, the effect of universalization of food supplements on immunization 
coverage can be predicted to be smaller. 


In interpreting these results, it would appear that the NHD approach has 
contributed significantly to the increased immunization coverage by the program 


A cen’ : — a 
Hhrerndhyse ser for the baseline, not shown in this paper, reflects what Figures 10 and 11 indicate: food receipt is only weakly 
umzation coverage. Again, this strengthens the likelihood that it is the organizing influence of NHD, which 


became more widely prevalent b ; 
th 
drives immunization rates y the endline, rather than food supplements without linkages with other health services, that 
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endline, acting through multiple ways. It has helped strengthen predictable 
outreach services where they were not happening earlier; has provided a 

formal forum for convergence with ICDS (and possibly local community groups) 
and thus increased the number of hands available to conduct immunization 
sessions. At least in some situations, the food incentive has contributed to 
greater participation in services at the NHD. Bigger gains can be expected by 
strengthening the quality of NHDs through effective use of counseling at the right 
time and tracking mechanisms to identify children due for immunization at each 
NHD. Another paper in this series, Supplemental Feeding, deals in greater detail 
with the different ways in which food supplements influence services. 


The issuing and retention of immunization cards are important indicators of the 
seriousness of the effort by the Health system and level of awareness among 
families. As seen in Table 7.1, program-wide, the proportion of mothers reporting 
that they had been issued immunization cards increased 20 percentage points 

to 73 percent by the endline, while the proportion actually able to produce an 
immunization card by the time of the survey increased by approximately 

15 percentage points. Card receipt either reached near-universal levels or rose by 
over 15 percent in all states other than Chhattisgarh. Card retention rates, however, 
are high only in West Bengal (82.5 percent), and increases in card retention range 
from 3.8 percentage points in Chhattisgarh to about 32 percentage points in Orissa. 
Thus, while there are clear signs of increased effort in most states, the use of 
immunization cards has become the norm only in West Bengal. 


Immunization Card 


Contacts, Advice and As part of the endline household survey, respondents (mothers) were asked 

Motivation whether they were aware of a NHD being held at the AWC or such an event 
happening in their village. In addition, they were asked whether any service 
provider or volunteer’? had contacted them over the previous one month and 
given information or advice related to immunization. They were also asked if, 
when the child was last immunized, someone had come to their home and asked 
them to take the child to the immunization session or if they had taken the child 
on their own initiative. Table 7.4 presents the responses, and compares complete 
immunization rates between those who answered each of these questions 
positively or negatively. As can be seen, each of these four factors shows a strong 
association with immunization coverage in almost all states other than West 
Bengal. The strongest association appears to be in Orissa, Rajasthan and Uttar 
Pradesh. In West Bengal, the only factor that may have had some influence is 
whether the child was called for immunization when last immunized. 


The possible programmatic importance of each of these individual factors may 
be estimated by calculating the likely increase in overall immunization coverage 
if the factor alone were applied universally to the population. One can predict 
that contacting all families to tell them about immunization is likely to yield 
increments in complete immunization coverage of about 18 percent points in 
Rajasthan or 21 percentage points in Orissa, and that other states are likely to 
gain less than 10 percent points by a similar initiative.” 
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Table 7.5: Influence of selected background variables on full immunization rates (card or recall), 12-23 months, EL (2006) 
able 7.5: 
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%FI = % fully immunized; *significant at 95% confidence level; **significant at 99% confidence level 
***significant at 99.9% confidence level 


Since information about these factors was not available from the baseline, it is 
not possible to provide comparative analysis of either the prevalence of these 
factors across the baseline to endline, nor of any change in the influence of these 
variables on immunization rates over the life of the program. 


\ 


In sum, it appears that many of the simpler operational issues related to provider 
practices seem to be well-grounded, but quality issues persist. As these are 
self-reported practices, it is difficult to assess the regularity and consistency of 
these practices without further survey or monitoring data. 


Background Factors 


Table 7.5 on pp. 24-25 provides evidence of the influence of selected background 
variables - a socioeconomic score (SES), sex of the child, maternal literacy, and 
two indicators of the distance of the AWC from the home of the respondent and 
shows change over time in the extent of their influence on immunization rates. 
The findings from a bi-variate analysis are: 


In general, the influence of these variables appears to have been stronger at 
the baseline than at the endline. 


In the case of two states, Uttar Pradesh and West Bengal, immunization 
coverage rates in boys were significantly higher than in girls at the baseline, 
but the differences disappeared by the endline. At the endline, there are no 
clearly discernible gender differences in any state. 


24  ——- RACHA Program 2001-2006 


—— 


BL 


= 


R / T T T 
fo FI N “%o FI % FI | | % FI | N 


428 64.8 3 


N 


‘ . 450 37.8 


43.0 | 348! 51. 


o66 205 16.4 171 28.0 37.0 | 158 
= ee ‘. 4 ds 1 | 
48.6 242 | 48.7 189 
-20.6"** 117" | Ts 9 


Ww 
wo 
% 
nN 
Ls} 
faa) 
og 
e 

N 
= 
—_= 
m 
> 
uo 
NS 
Ss 
N 


42.8 250 | 40.2 ; 185 


46.4 


5 ° 
[aero | | |e 
5) 70.4 
SPPeR NSTRIET GRis [ 
rere 


e Over the life of the project, immunization coverage increased amongst 
children of educated as well as uneducated mothers. Larger increments in 
immunization coverage were observed in children of uneducated mothers in 
Andhra Pradesh, Orissa and West Bengal, while in Madhya Pradesh, Rajasthan, 
and Uttar Pradesh, larger increments were seen in children of mothers with 
at least primary education. Chhattisgarh and Jharkhand showed marginal 
differences in the immunization coverage rates between the two groups. 


e With respect to SES, larger increments in immunization coverage were 
observed in the lower half of the SES in Andhra Pradesh and West Bengal, 
whereas Madhya Pradesh and Rajasthan showed larger increments amongst 
the upper half of the SES. Between the baseline and the endline, the 
difference between the lower and upper SES halves narrowed more than 
10 percentage points in Andhra Pradesh, Chhattisgarh, Uttar Pradesh and 
West Bengal, but increased by more than 10 percentage points in Madhya 
Pradesh and Rajasthan. The differential increments amongst the lower half 
and the upper half of the SES indicate that INHP-II partially succeeded in 
influencing immunization service provision to the most marginalized (per 
the SES). At the endline, however, the immunization coverage in the lower 
half of the SES remained significantly lesser than among the upper half 
lower in four states (Madhya Pradesh, Orissa, Rajasthan and Uttar Pradesh). 
More focused effort is required to close the equity gap between the two 


groups in accessing services. 
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In most instances, the distance of the home from the AWC does seem to 
matter, but the differences in immunization coverage rates between those 
staying close to the AWC or far from it are small, becoming statistically 


significant in only a couple of states. 


Multi-variate analysis should help to further discern the relative influence of the 


above factors on immunization rates. 


in Improving Immunization Coverage: 


Overall, it is clear that available evidence does not permit a thorough and robust 
analysis of the influence of several important program elements. For instance, 
while at an aggregate level, evidence indicates that tracking and delivery 

systems improved, in the absence of specific studies to capture variations across 
villages, sectors and blocks and their links to immunization coverage rates, it is 
difficult to definitively say whether tracking and delivery systems were indeed 
strengthened across areas. Similarly, while vaccine supplies were by and large not 
a major concern in most states in the period of reference, specific instances of 
supply disruptions that could have influenced immunization rates have not been 
documented. Equally, disruptions in cold chains and local distribution mechanisms 
have not been examined for want of documented evidence. It should be 
recognized, of course, that collecting objective evidence related to such disparate 
program elements is not a simple task, even at small scale, and there are hardly 
any studies that establish one-to-one links between such elements. 


In INHP-II, the choice of interventions prioritized were based on an analysis 

of the major operational problems of the immunization program at the end of 
INHP-I. An unacceptably large proportion of children were left out of services in the 
baseline in many states, and complete immunization rates remained low. While most 
communities had overcome the instinctive fear of child immunization programs 

by then, this had not translated into proactive attempts by communities to seek 
immunization services. There were still large gaps in the Health system’s drive and 
ability to sustain a minimal level of reliability in immunization services and back-up 
assurance in case of problems. Thus, the prime focus of the program was to make 
immunization services more regular and accessible and to improve their utilization. 


Information from the baseline and lessons learned in the program reinforced 
the need and benefit of strengthening day-to-day operations at the field level. 
For instance, while supply disruptions were seen at several points during the 
Program in many states with respect to one or other antigen, this was not 

the bottleneck in most instances. Inadequate utilization of available services 
due to failure to implement long-established norms of outreach immunization 
Programs was seen to be the main problem. Casually designed and maintained 
recording and tracking mechanisms, often not user-friendly, were also a 
common problem. Simple directives such as the need to open fresh vials of 
vaccines when only a few children were present were not followed. Supervisory 
mechanisms were cursory and inadequate to ensure that planned immunization 
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sessions actually happened, and failed to detect and correct glaring instances of 
program weaknesses, including excluded families and communities in the — 
network, Real denominators were almost never used for monitoring programs i 
and deliberate over-reporting of coverage by program managers at the block or 
higher levels was common. While the ANM and the AWW regularly exchanged 
information and assistance, the process lacked rigor and was poorly supported by 
program managers of the two departments at higher levels, largely due to lack of 
coordination and blaming each other for inadequate performance. Years of focus 
on monitoring processes without objective evidence of progress on outcomes 
had blunted the ability and drive to seek results. A standard response at higher 
levels of the bureaucracy was that services were in place, but there was no 
demand for services. 


The sharing of results from the RAPs in the panel districts in early 2004 at the 
district level provided opportunities to test the will of systems to turn things 
around at a stage when the program was just beginning to reach scale and work 
directly with program implementers in the ICDS and Health Departments. The 
initial indignation at the “unrealistically low” estimates of immunization coverage 
found in the surveys soon gave way to willingness to sit together and plan for 
greater effectiveness, at least in these eight districts for which objective coverage 
figures were available. Other districts later also conducted joint planning exercises 
involving Health and ICDS, using available data. A key feature of these planning 
exercises was that they planned for other major activities in maternal and child 
health, along with increasing immunization coverage. This enabled realistic plans 
to be drawn up. Another feature was the customization of plans for local contexts 
such as access, and staff vacancies. These plans focused attention on addressing 
basic operational issues such as the regularization and simplification of register 
formats, where feasible; joint generation of due-lists by AWW and ANM; joint 
reviews at sector and block levels using coverage data with denominators; and 
fresh emphasis on making fixed-day, fixed site immunization possible by revisiting 
operational problems of ANM schedules. Additional training needs were also 
identified and addressed. 


As this approach was implemented district by district, involving different sets of 
system staff and INHP teams, there was considerable variability in the pace and 
intensity of progress across districts. There were some clear instances of these 
processes yielding immediate results. For instance, in Bankura, the panel district 
of West Bengal, immunization coverage with most antigens was found to be 
very high, and yet was falling back in complete immunization coverage because 
measles immunization was low. This finding was addressed intensively over one 
year, and when the second round of assessments was conducted, and measles 
coverage (and full immunization) was seen to have increased substantially. In 
other districts, changes were slower to come by. 


Other lessons were also being learnt. Data from the first round of RAPs found that 
receipt of food rations was only weakly associated with better service coverage 

in most states. The convergence of food rations with predictable health spoil 
was critical as was the attention to simple managerial issues in conducting 
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immunization sessions. This led to greater emphasis being placed on the 
operational issues of immunization sessions, even where ensuring the NHD was 
not easy. Another “best practice” that INHP-II began with was the use of social 
maps to identify left-outs and monitor drop-outs. It soon became apparent that 
while it served a useful purpose as a one-time tool to ensure that no community 
or cluster of homes was “falling through the cracks”, a map was not the preferred 
mode of tracking service coverage for either the AWW or the local women’s 
groups. Being local, they did not seem to need maps for this purpose. In fact, 
except where the AWW had a high level of formal education, even the use of 
service registers (such as the immunization register) was largely more beneficial 
for reporting than for tracking; they tracked children from memory rather than 
determining “due” children on the basis of immunization dates entered in the 
register. The main bottlenecks that prevented effective tracking were ensuring 
that a child got on to the immunization register soon after birth, that efforts 
were made to actually draw up and follow “due lists” and to remind specific 
families about the impending immunization session where they must bring their 
child. As it became increasingly clear over time, joint ICDS-Health Department 
reviews at sector and block levels were one of the more effective ways of tracking 
whether planned immunization sessions actually took place. 


In comparison to service quality issues, “demand” side problems were not as 
significant. While every community had some families who refused vaccinations, 
these were the exception, not the rule. By and large, awareness and acceptance 
of immunization was high. In unexpected corners, one came across examples 

of heightened awareness, such as mothers in relatively remote communities 

in Jharkhand routinely queuing up for immunization with freshly purchased 
disposable syringes and needles because they found the reusable needles used 
by the ANM too blunt and painful for their children. It is also quite common 

in communities with relatively high coverage rates for mothers to arrive at the 
fixed-day, fixed-site immunization sessions without being informed or reminded. 
Field observations indicate that a high level of community participation is a 
logical outcome of regular and reliable services, not the other way around. Visibly 
regular and predictable services are more effective in obtaining “community 


participation” than intensified behavior change communication in the face of 
poor services. 


In addition to these efforts to seek and solve simple operational problems, the 
Program also supported several initiatives of UNICEF and other development 
partners in different states, such as the facilitation of micro-planning of 
immunization sessions, promotion of injection safety measures and, refresher 
training for cold chain maintenance. There were other areas requiring obvious 
reform, such as the management information system, and several attempts were 
made to influence such reform at state and national levels, independently or 
along with other agencies. However, the nature of these issues was such that it 
was not realistic to expect to see changes in outcomes on the ground within the 
life of the program as a direct result of these initiatives. CARE was also an active 
participant in the national UIP review in late 2004. The extensive report of that 
review echoes several lessons learned in the course of the INHP. 
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Conclusions 


The INHP attempted to help the ICDS and Health Departments of the Government 
of India bring about sustainable change in primary immunization coverage rates 
across 75 districts in eight states between 2001 and 2006. The primary focus was 
on improving field operations at sub-district levels, converging efforts of ICDS 
and Health and promoting mechanisms to maximize outreach, especially to get 
the “left-outs” into the service fold. This paper examined available evidence to 
document change in immunization coverage rates and factors that influenced 


the change. 


There was a significant increase in the coverage rates of individual antigens as 
well as full immunization with variations in the increments across states. While a 
few states continue to have significant numbers of children entirely unvaccinated, 
drop-out rates have reduced substantially across the program. Also, there has 
been a reduction in the numbers of children entirely unvaccinated, or the left- 
outs. While immunization rates continue to be higher among the upper half of the 
SES in half the states, significant reductions were seen in differentials between 
the upper and lower halves of SES in several states. Variations across states and 
districts can probably primarily be ascribed to the strengths and weaknesses in 
delivery systems and the focus and intensity of INHP implementation. Rajasthan 
and Uttar Pradesh are the two states that will require special attention. 


The pace and extent of the increase in immunization coverage (children fully 
immunized) is greater in INHP-II areas over a five year period compared to the 
increase in overall state estimates (rural) over an eight year period between NFHS-2 
and NFHS-3, with the exception of Chhattisgarh. NFHS data (rural) for the INHP-II 
states can serve as a reasonable comparison, given the absence of true controls. 


One of the approaches of INHP, the NHD, appears to have contributed to the 
improved coverage. NHDs helped strengthen systems, encouraged regularity 

and predictability of outreach immunization sessions, converged efforts of 

the AWW and the ANM and mobilized community volunteers, community 

based organizations and local governance institutions i.e., the Panchayati Raj 
Institutions, around getting children immunized. The NHD also explicitly linked 
supplementary feeding in the form of take-home rations to health service delivery, 
thus facilitating food distribution to serve as an incentive to avail services. A 
number of related elements, including training of functionaries, use of improved 
tracking mechanisms to ensure universalizing services, e.g., the catchment 

area approach, drawing up “due lists ” prior to each immunization session, use 

of data from internal information systems and from independent assessments, 

and the use of convergence mechanisms between ICDS and Health undoubtedly 
helped in extending outreach and reducing left-outs and drop-outs. For improving 
effectiveness of NHDs as they get further replicated, it is important to ensure that 
these elements form an integral part of the core minimum NHD package, along 
with the distribution of rations and immunization. 


sary registration of preqnant women at the AWC, presumably for supplementary 
feeding had a positive association with full immunization coverage. This 
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highlights the importance of universally getting pregnant women into the servi 
fold and tracking the mother-child pair for the complete set of health ange 
S. 


The INHP experience indicates that weaknesses of supply gaps and the demand 
side issues are actually not the bottlenecks that limit high coverage. The lack of 


attention to simple tracking processes and poor supervision, and unsatisfactor 
provider knowledge and skills are bigger challenges. ; 


The achievements and challenges described here are ultimately reliant upon in 
quality of existing government systems, and the approaches used could best 
be sustained by reorganizing existing resources to improve effectiveness and 
efficiency and using well-established principles of Organizing immunization 


services. The INHP merely played a catalyst role in establishing or improving 
such processes. 


One lesson that has been repeatedly learned in immunization programs globally 
is that hard-earned gains can be quickly lost when one or other of the many 
elements that sustain such programs are neglected. The next steps must be to 
find ways of helping key delivery mechanisms become established and respected 
traditions within public health programs, with sufficient internal and external 
checks and balances to ensure timely identification of slippages and their 
rectification. Now that improved coverage is being established, it is time to 
pay attention to quality and ensure that immunization translates into effective 
protection and reductions in child morbidity and mortality. For many of these 
states, the battle has only begun. 


Endnotes 


i The shifting of universe between baseline and endline in the case of Jharkhand/Bihar and 
Madhya Pradesh/Chhattisgarh in the RACHNA program has been discussed in the paper entitled 
Methods used for Assessments. For NFHS as well, the creation of new states between the second 
and third rounds resulted in problems of comparability across rounds. Some of the relevant 
issues are as follows: 


¢ Only sparse summary estimates of immunization coverage for Chhattisgarh and Jharkhand 
are available, since these had to be derived from NFHS-2 data for erstwhile Madhya 
Pradesh and Bihar, and these derivations have not yielded representative results. Also, 
rural/urban disaggregated figures are not available for these states for NFHS-2, and the 
estimates of increments in coverage between NFHS-2 and NFHS-3 are not based on rural 
estimates but on overall state estimates (while the figures for the other states are all . 
based on rural estimates). Absolute coverage for NFHS-3 as depicted in Figures 3 to 5 is 
for rural areas, however, for all states including Chhattisgarh and Jharkhand. 

e Inthe case of Madhya Pradesh, the three CARE-assisted districts (Seoni, Chhindwada 
and Balaghat) are known to be among the most poor-performing districts of the state, 
bordermg Chhattisgarh, and thus are atypical of the state as 4 whole. The RACHNA 
baseline for Madhya Pradesh is also not representative of the current three Madhya Pradesh 
districts, since most of the area covered by RACHNA then is now in Chhattisgarh. Thus, 
although Chhattisgarh and Madhya Pradesh share the same baseline for RACHNA, this 
baseline is more true for Chhattisgarh than for Madhya Pradesh. It is difficult to find a 
representative baseline for the current RACHNA districts of Madhya Pradesh. 

© In the case of Jharkhand, the large percentage increment in NFHS starts from a very low 
estimate for the baseline (8.8 percent), while the hig 
mainly from areas of Bihar that were a part of INHP-I 
program universe for INHP-II. 


her baseline for RACHNA comes 
but were later dropped from the 


points, between coverage rates 


i i ; of not more than a few percentage 
ii There are differences, usually on of the RAHN 


reported in this paper and those found in the report on the Final 
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program (May 2006). These differences arise from two main reasons: further cleaning of endline 
survey data and differences in definitions used for deriving estimates. The Final Evaluation 
report carried estimates for a limited number of outcome variables that represented the results 
committed by the program to the donor. For instance, among all antigens, only tetanus toxoid 
and measles vaccine coverage were reported in the Final Evaluation report. These estimates 

had to follow definitions (numerators, denominators, exclusions, etc.) that closely matched 

the definitions used at the baseline in 2001 (some of which again had a historical base in 

the INHP-I baseline of 1996). The current paper makes no attempt to restrict itself to those 
definitions, but uses the most robust and globally acceptable definitions and strives to 

maintain thorough internal consistency. Thus, equivalent definitions are applied to baseline and 
endline data. In sum, both, the baseline-endline comparisons in the RACHNA Final Evaluation 
report and the baseline-endline comparisons in this paper remain valid, and any numerical 
differences in indicator values between the two reports should not be construed to be errors. 


These estimates, based on Table 7.3, are derived by subtracting average immunization rates 
for the full sample from the immunization rates among those who report that they had 
received food supplements. This difference will be large in cases where the proportion of 
non-food-recipients is large, and the immunization rates in food recipients is much larger 
than in non-recipients. This is true particularly of Rajasthan. This is explained further in 
the next endnote. ; 


As in the case of food supplements discussed earlier, the estimates are derived by subtracting 
the immunization rates in the population from the immunization rates among those that 

were exposed to the factor under consideration, such as those who were aware of NHD or 
those who were contacted and counseled about immunization in the previous month. This 

is the equivalent of population attributable risk, and will be higher when the number as yet 
unexposed to the intervention is still high, and there is a large difference in the likelihood of 
getting immunized among those exposed and not exposed to the intervention. This could have 
important programmatic implications. For instance, in Orissa, the difference in immunization 
coverage among those counseled and contacted in the previous month is 31.5 percent 

more than those not contacted and counseled, and the difference among those called for 
immunization and not called is 30.8 percent. The influence of the two factors appears similar. 
However, currently, just about a third of children are being contacted and counseled about 
immunization, while 92 percent are being called for immunization, presumably on the day there 
is an immunization session in the village. By ensuring that all children/families are contacted 
and counseled about immunization, there is likely to be a gain of about 20 percentage points 
in average immunization rates in Orissa, but by ensuring that everyone is merely called for 
immunization the same day is likely to yield an increase of merely 2.5 percent. From a program 
perspective, it would be more beneficial therefore to push for an increase in the proportion of 
families contacted and counseled. Multivariate analysis controlling for a number of variables 
could help make more refined estimates of the strengths of such influences. 


(Contd. from inside of the front cover) 


in at least some of the states, the equity gap for immunization covera 
endline, the majority of children in seven of the eight states were rec 
level of coordination that can be achieved between ICDS and health 
families to access services. 


ge narrowed over the project life. By the 
eiving vaccines at the AWC, indicating the 
Programs to make it more convenient for 


Several key factors appear to have contributed to positive change: the organizing influence of NHD where 
the Anganwadi Workers (AWW) and the Auxillary Nurse Midwife (ANM) come together to provide services, 
the active involvement of ICDS and the AWW that made tracking children more feasible, the incentive- 
function of food supplements given out at the AWC along with immunization, the attention paid to the use 
of registers and other tracking systems, greater outreach of service providers to homes to inform and advice 
about immunization, the regular joint reviews between ICDS and health officials at block and district levels, 
and the use of coverage data generated independently of routine reporting for the purpose of monitoring 
and planning. Variations across states and districts can probably primarily be ascribed to the strengths 

and weaknesses in delivery systems and the focus and intensity of INHP implementation. This experience 
suggests that simple but consistent inputs can lead existing systems to achieve significant improvement in 
immunization coverage at scale. 
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This series of working papers was envisioned and written by persons actively involved in the program 
design and implementation. USAID/BASICS directly contributed to the writing and production of this 
series of papers in several ways before it closed in India in December 2007. The IMMUNIZATIONBasics 
projects reviewed several drafts and provided valuable comments and suggestions. A number of data 
support and field staff gave invaluable contributions, and the papers were reviewed by CARE-India and 
USAID/India staff. 


The main contributors to this paper were Amit Chakraverty, Ashi Kathuria, Biraj Laxmi Sarangi, 
Joby George, Manish Mathur, Sridhar Srikantiah and Usha Kiran. 


Questions and comments are welcome. For this paper, they may be addressed to 
sridharmfc@yahoo.com or to dora@careindia.org. 
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About RACHNA 

Two major projects of the Reproductive and Child Health, Nutrition and HIV/AIDS (RACHNA) program 
of CARE-India completed five years of work supported by funds from USAID in late 2006. The 

second phase of Integrated Nutrition and Health Project (INHP-II) was aimed at helping reduce 

child malnutrition and mortality. The rural component of the Chayan project primarily addressed the 
unmet need for spacing methods, while its urban component attempted to reduce HIV transmission 
among at-risk groups. Together, the projects covered 78 districts and 22 cities, spread over 10 states, 
and worked closely with key national programs and a spectrum of different partners. This series of 
working papers documents the results and lessons from these five years. 
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